Statistical Learning of LQG Controllers

The identification of the cost function, the optimization of which generates the optimal behavior of a dynamic system, is one of the most challenging problems in the areas of sensorimotor control, machine learning and control theory. The problem could be state as follows: given the dynamics of a system and by observing its behavior we ask what is the cost function that if it was optimized it would generate the observed dynamic behavior. In the optimal control theory framework the problem of estimating the cost has been first addressed by Kalman [1]. In this work, the quadratic cost was recovered under the assumption that the weight assigned to the control quadratic term of the cost function is a scalar. In [2] author solve the inverse problem in optimal control by estimating the weight matrices Q and R using LMIs. This work is an extension of the work by Kalman since there is no assumption regarding the form of the weight matrices incorporated in the quadratic cost function. Both previous approaches address the inverse problem of optimal control for the case of deterministic linear dynamic systems. In the machine learning community the problem of deriving the cost function by observing behavior has been mainly addressed insight the Reinforcement Learning framework. More precisely in [3] , a robot learns how to execute a demonstrated trajectory. The cost function used for this task is quadratic with respect to the error measured given the desired trajectory. This approach could be used only in cases where the planning phase of task consists of the definition of a desired trajectory. In human motor tasks such reaching movements the biomechanical redundancy and the trial by trail variability has been successfully explained [4], [5] without assuming the existence of any desired trajectory (minimum intervention principle). Moreover, the existence of a desired trajectory leads to a suboptimal motor behavior based on the biomechanical and noise characteristics of the human motor system.